A 3D image of an object can be generated from one or more two-dimensional (2D) images using various reconstruction techniques. For example, multi-view geometric reconstruction methods, such as structure-from-motion (SfM) and simultaneous localization and mapping (SLAM), recover point clouds as the underlying 3D structure of red-green-blue (RGB) image sequences, often with high accuracy. Point clouds, however, lack inherent 3D spatial structure for efficient reasoning. For this reason, at least in some scenarios, mesh representations are more desirable than point clouds for 3D object reconstruction. A 3D mesh is a structural representation, or model, of a polyhedral object, where the three-dimensional reference points of polygons (x, y, and z) in the model define the height, width and depth of various object surfaces. Meshes are significantly more compact as data structures than point clouds because meshes have inherent geometric structures defined by point connectivity, while they also represent continuous surfaces that are useful for many applications, such as texture mapping. However, as will be further explained herein, meshing point clouds is a difficult and computationally expensive problem, and existing solutions for meshing point clouds are impractical.
Another limitation of multi-view geometric methods is that they rely on hand-designed features and can be fragile when assumptions about those features are invalid for a given image. This happens especially in texture-less regions or with illumination variations. By contrast, data-driven approaches include prior knowledge of shapes that are likely to be in a given image (also referred to as shape priors) for solving ill-posed 3D reconstruction problems. Such data-driven approaches have been applied to 3D prediction tasks using single images. However, these approaches can only reliably reconstruct from the known space of training examples used for learning, resulting in a limited ability to generalize to unseen (non-learned) data.
Therefore, complex and non-trivial issues associated with 3D object reconstruction remain due to the limitations of these existing techniques.